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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.1 14, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eUgible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.1 14. Applicant's submission filed on April 7, 2008 has been entered. 

Response to Arguments 
As to applicant's arguments regarding the newly amended limitation "a sequence of in 
vivo images in a same set," applicant argues the images disclosed by Wyman reference are from 
two separate sets. In examination the claims are given the broadest reasonable interpretation. 
The claims do not recite any limitations defining what a "set" is nor does the disclosure of the 
present invention explicitly present a definition for a "set." Therefore, a "set" of images may 
reasonably be construed as a group images related in some way. The Wyman reference discloses 
registering a sequence of MRl images to CT images or images obtained from the same imaging 
modality but at different times (col. 7, lines 30-60; col. 1, lines 53-65), which are related because 
they are of the same physical structure. Therefore Wyman discloses "a sequence of in vivo 
images in a same set" as required by claim 1. Regardless, Wyman fiirther discloses the disclosed 
technique can be used for registering any sets of 2D or 3D images (col. 7, lines 61-64) and it is 
therefore obvious to one of ordinary skill in the art at the time the invention was made that the 
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techniques taught by Wyman could be used to align the images obtained from the encapsulated 
endoscope system such as the one taught by Uchiyama as the examiner had clearly articulated in 
the final Office action (pages 7 and 8). In this case, the individual images obtained by the 
encapsulated endoscope correspond to the "sets" to be aligned taught by Wyman. However, 
these images have all been obtained by the encapsulated endoscope in as it travels through the 
patient and is therefore "a sequence of in vivo images in a same set" as recited in claim 1 and 
described in the detailed disclosure of the present invention. 

As to applicant's arguments regarding the "accumulated rotation angle," applicant 
submits Wyman does not teach or suggest accumulating a rotation angle and using it to correct 
other images in the sequence. Upon further consideration Wyman does not disclose in enough 
detail the registration and alignment of any additional image to sufficiently anticipate the 
accumulated rotation angle as recited in claim 1. However, Uchiyama clearly discloses 
determining an accumulated rotation angle between a reference image and one of the other 
images and using it to align the multiple images obtained from the encapsulated endoscope (Figs. 
5 and 6; pages 3-4, paragraphs 58-73). Thus, a combination of the teachings of Wyman and 
Uchiyama would render the claimed invention obvious. This combination will be articulated in 
detail below. 

Regarding arguments presented for claim 8, Wyman selects points that do not reside on 
the axis of the first image plane. Wyman exemplarily selects the comer points because they 
would have the greatest amount of movement between the two images if they are rotated with 
respect to one another (col. 10, line 48 - col. 11, line 3). However, in view of applicant's own 
specification describing the selection of feature points and determining the rotation angle based 
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on the movement of the points between the two images (page 12, line 26 - page 13, line 8), 
examiner beheves the selection of the points on the axis to be an obvious design choice. The 
specification at page 13, lines 3-4 clearly states "people skilled in the art can easily extend this 
analysis to points that are not on the axis." Examiner believes a selection of points on an axis in 
addition to the comers of the images in invention of Wyman would not cause Wyman's 
technique to fail in aligning the images. 

Regarding arguments presented for claim 17, the claim language does not explicitly 
preclude the images from spatially overlapping and therefore does not exclude the images taught 
by Wyman and clearly not the endoscope images taught by Uchiyama. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 1 02 ol' this title, if the differences between the subject matter sought to be patented and the prior art are 
such thai the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinaiy skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1, 3-9 and 11-17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Wyman in view of US 2003/0229268 to Uchiyama et al. (Uchiyama). 

Regarding claims 1 and 17, Wyman discloses a digital image processing method for 
automatic axial rotation correction of a sequence of in vivo images in a same set (col. 6, lines 34- 
37; col. 7, lines 30-64; col. 1, lines 53-65), comprising the steps of: 
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a) selecting, as a reference image, a first arbitrary in vivo image from the sequence of in 
vivo images (col. 6, lines 40-45); 

b) determining a rotation angle between a second arbitrary in vivo image selected from 
the sequence of in vivo images and the reference image of the sequence (col. 6, lines 57-61; col. 
8, lines 23-37; col. 9, lines 43-49; col. 10, line 48 - col. 11, line 3; col. 11, lines 41-58, Wyman 
teaches applying qffine transformations including rotating the images in the image registration. 
The Examiner believes the determination angle of a rotation angle between the reference image 
and the second image is inherent to the rotation transformation of registration method taught by 
Wyman.); 

c) correcting the orientation of the second arbitrary in vivo image, with respect to 
orientation of the reference image and corresponding to the rotation angle (col. 6, lines 61-64; 
col. 8, lines 23-37; col. 9, lines 43-49; col. 10, line 48 - col. 11, line 3; col. 11, lines 41-58). 

Wyman further discloses using orientation corrected images to determine a rotation angle 
between other selected in vivo images and the reference image (col. 6, lines 57 - col. 7, line 3; 
col. 8, lines 23-37; col. 9, lines 43-49; col. 10, line 48 - col. 11, line 3; col. 11, lines 41-58; col. 9, 
lines 1-6, Wyman teaches applying qffine transformations including rotating the images in the 
image registration thereby bringing an arbitrary number of images, into alignment. Wyman 
further teaches the transformations may be applied to a subset, convergence is determined and 
the steps are repeated, thus the transformed subset (orientation corrected images) is used to 
perform registration of the rest of the set (other in vivo images). The Examiner believes the 
determination angle of a rotation angle between the reference image and the second image is 
inherent to the rotation transformation of registration method taught by Wyman.); and correcting 
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for the other selected in vivo images that do not match the reference image's orientation and 
where there exists a rotation angle between the other selected in vivo images and the reference 
image (col. 6, lines 61-64; col. 8, lines 23-37; col. 9, lines 43-49; col. 10, line 48 - col. 11, line 3; 
col. 11, lines 41-58; col. 9, lines 1-6, Wyman teaches the process may be extended to bring an 
arbitrary number of images or image sets into alignment.). Wyman further discloses the 
disclosed technique can be used for registering any 2D or 3D images (col. 7, lines 61-64) 

While Wyman discloses bringing any arbitrary number of images into alignment, the 
explicit details of the alignment are limited to a discussion of only two images. Therefore, 
Wyman is silent with regard to an accumulated rotation angle which is used to correct other 
selected in vivo images and the reference image. Uchiyama discloses a using orientation 
corrected images to determine an accumulated rotation angle between other selected in vivo 
images in a sequence and a reference image from a sequence of in vivo images captured by an 
encapsulated endoscope and correcting for the other selected in vivo images that do not match 
the reference image's orientation using the accumulated rotation angle where there exists a 
rotation angle between the other selected in vivo images and the reference image (Figs. 5 and 6; 
pages 3-4, paragraphs 58-73). 

Wyman and Uchiyama are combinable because they are both concerned with medical 
image processing, specifically rotating the medical images to properly align them for display. It 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
combine the techniques taught by Wyman and Uchiyama such that the image registration 
techniques taught by Wyman could apply to images captured by an encapsulated endoscope 
system such as the one taught by Uchiyama (and the present invention) and that the method of 
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Wyman could be modified to include using the orientation corrected images to determine an 
accumulated rotation angle between other selected in vivo images the sequence and a reference 
image from a sequence of in vivo images captured by an encapsulated endoscope and correcting 
for the other selected in vivo images that do not match the reference image's orientation using 
the accumulated rotation angle where there exists a rotation angle between the other selected in 
vivo images and the reference image, as taught by Uchiyama, in order to bring any arbitrary 
number images into alignment with the reference image. 

Regarding the limitation "where the second arbifrary image and the reference image need 
not have a spatially overlapping area" as recited in claim 17, the language of the claim does not 
explicitly preclude the images from a having spatially overlapping area, just that they don't 
necessarily have to. Thus the images disclosed by Wyman meet the requirements of the claim. 
Further Uchiyama' s images are a sequence of image captured by the encapsulated endoscope and 
are aligned based on the accumulated rotation angle and therefore, the first image would not need 
not to be spatially overlapping with an arbitrary other image in the sequence. 

Regarding claim 3, the combination of Wyman and Uchiyama further discloses the step 

of correcting the orientation of the arbitrary in vivo image, with respect to orientation of the 
reference image and corresponding to the rotation angle uses an accumulated correction angle 
derived from the accumulated rotation angle (Wyman, Fig. 5; col. 8, line 60 - col. 9, line 6; col. 
10, line 48 - col. 11, line 3; Uchiyama, Figs. 5 and 6; pages 3-4, paragraphs 58-73). 
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Regarding claim 4, the combination of Wyman and Uchiyama further discloses the 
rotation angle measured with respect to an optical axis of an in vivo camera used to capture the 
plurality of in vivo images, and wherein the optical axis is perpendicular to an image plane and is 
parallel to the in vivo camera's travel trajectory derivative (Uchiyama, Figs, 3,9, 10, 12, 13, 14, 
15, 16; paragraphs 6, 42, 55-58). 

Regarding claims 5 and 6, the combination of Wyman and Uchiyama further discloses 
the rotation angle is defined in a right-hand system or a left-hand system and angle is rotated 

counter-clock wise or clockwise relative to the reference image's orientation, such that the 
rotation angle is a signed value (Wyman, Fig. 8; Fig. 9E; col. 8, lines 1- 37; col. 11, line 41 - col. 
12, line 4; col. 14, lines 60-62). 

Regarding claim 7, the combination of Wyman and Uchiyama fiirther discloses the 
plurality of in vivo images have a plurality of feature points, and wherein the plurality of feature 
points are used for finding an orientation difference between two in vivo images (Wyman, col. 6, 
lines 46-64). 

Regarding claim 8, Wyman discloses an origin of a two-dimensional coordinate system 
of the in vivo images, thus defining an image plane, is at an image's center (Fig. 5; col. 10, lines 
48 - col. 11, line 3, Wyman clearly shows two-dimensional images oriented in an X-Y 
coordinate system, with Z coordinate identifying each image's two-dimensional plane location in 
the stack of images, with an origin at the intersection of the intersection of axes. It is clear if the 
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image sets are limited to a single image (as suggested by Wyman at column 8, lines 63-65), the 
coordinate system would be limited to a two-dimensional system as there is no "stack" of 
images"), and further comprising the steps of: 

a) collecting the plurality of feature points that reside on a first image plane (Fig. 5; col. 
6, lines 46-54; col. 10, line 48 - col. 11, line 3, Wyman selects points at the comers of the 
images); 

b) finding a corresponding plurality of feature points in a second image plane (col. 6, 
lines 46-54; col. 10, lines; col. 10, line 48 - col. 11, line 3); 

c) determining whether a feature point that resides on the first image plane moves in the 
second image plane (col. 6, lines 54-57); and 

d) measuring the feature point's movement in the second image plane to determine the 
rotation angle and its direction (col. 6, lines 57-64; col. 10, line 48 - col. 11, line 3). 

The combination of Wyman and Uchiyama does not disclose expressly collecting the 
plurality of feature points that reside on the axis of a first image plane; determining whether the 
feature point that resides on the axis of the first image plane moves off axis in the second image 
plane; and measuring the feature points movement off axis in the second image plane to 
determine the rotation angle and its direction. 

At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to modify the feature points taught by Wyman to include points that reside on the axis of 
the image plane. Applicant has not disclosed that collecting feature a plurality of feature points 
that reside on an axis of an image plane provides an advantage, is used for a particular purpose or 
solves a stated problem. One of ordinary skill in the art, furthermore, would have expected 
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Applicant's invention to perform equally well with points not on the axis because applicant's 
specification recites "people skilled in the art can easily extend this analysis to points that are not 
on the axis" (page 13, lines 3-4). 

Therefore, it would have been obvious to one of ordinary skill in this art to modify the 
combination of Wyman and Uchiyama to include collecting a plurality of feature points that 
reside on an axis of a first image plane to obtain the invention as specified in claim 8. 

Regarding claims 9, 11-16, the combination of Wyman and Uchiyama further discloses a 
computer storage medium having instructions stored therein for causing a computer to perform a 
method (Wyman, col. 11, lines 18-39). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ANTHONY MACKOWEY whose telephone number is 
(571)272-7425. The examiner can normally be reached on M-F 9:00-6:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella can be reached on (571) 272-7778. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Matthew C Bella/ 

Supervisory Patent Examiner, Art Unit 
2624 
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4/25/08 



